1. Field of the Invention
The present invention relates to an image display device comprising a matrix display device such as an LCD (liquid crystal display), PDP (plasma display panel), or DMD (digital micro-mirror device) and to an image display method, and more particularly relates to an art for enlarging and reducing images for displaying image signals on a display device having a higher or lower number of pixels.
2. Description of the Related Art
First, description will be made regarding image scaling processing as an example of image enlarging processing, following which description will be made of memory control accompanying processing in the vertical direction.
Here, an x-y orthogonal coordinates system will be used to describe the relation between the original image and the image following the enlarging processing (post-conversion image) with the x coordinates in the horizontal direction and the y coordinates in the vertical direction. The original image is sampled in the horizontal direction and vertical direction, and is made up of image data which is m pixels in the horizontal direction and k pixels in the vertical direction.
Let us consider a case of taking the image data made up of the m×k pixels and enlarging the data into image data made up of M×K pixels, M pixels in the horizontal direction and K pixels in the vertical direction.
FIG. 1 is a diagram schematically illustrating the relation between the original image and the image following enlarging conversion (post-conversion image). In the enlarged example of this image, the original image made up of m×k pixels is converted into an image made up of M×K pixels. In the drawings, the white dots indicate pixel data of the original image. The solid dots in the post-conversion image indicate image area in the post-conversion image provided based on the white dot pixel data in the original image. In this case, the ratio of enlargement in the horizontal direction is M/m, and the ratio of enlargement in the vertical direction is K/k.
In order to study the relation between the pixels of the original image and the pixels of the post-conversion image, reverse mapping will be performed wherein the coordinates of the post-conversion image are correlated to the coordinates of the original image. FIG. 2 is an example of reverse mapping of the pixel D in the post-conversion image to the coordinates in the original image.
In this figure, the white dots indicate pixels in the original image, and the solid dots indicate pixels in the post-conversion image which has been reverse-mapped. For the sake of simplicity, let us say that the pixels of the original image are adjacent by a distance of 1 in the horizontal direction and the vertical direction, with the pixel data being represented by the format of d(x, y). Here, x and y are integers.
In this example, an interpolating pixel D of the post-conversion image is reverse-mapped to coordinates wherein an area consisting of the four surrounding points of d(x, y), d(x+1, y), d(x, y+1), and d(x+1, y+1), is divided by the ratio of p:1−p in the horizontal direction and q:1−q in the vertical direction, wherein 0≦p<1 and 0≦q<1 hold. In this case, the coordinates of the reverse-mapped interpolating pixel D are expressed as (x+p, y+q).
In the event of a linear interpolating filter for example, the above p and q, and the four surrounding points of d(x, y), d(x+1, y), d(x, y+1), and d(x+1, y+1) around the interpolating pixel D yield the pixel data of the interpolating pixel D by calculation expressed as the following Mathematical Expression 1.D=(1−p)·(1−q)·d(x,y)+p·(1−q)·d(x+1,y)+(1−p)·q·d(x,y+1)+p·q·d(x+1,y+1)  [Mathematical Expression 1]
Now, description will be made regarding the scaling processing of the image in the vertical direction. Here, we will study an example of enlarging processing in the vertical direction for converting five lines into eight lines, as an example for description.
FIG. 3 is a diagram illustrating the manner in which coordinates in the post-conversion image are correlated to coordinates in the original image, with regard to the vertical direction. In the figure, the white dots and d represent image data of the original image, and d(0) represents image data on the line 0 in the original image.
Pixels d(0), d(1), d(2), and so forth of the original image are adjacent by a distance of 1. Also, the solid dots and D represent image data of the post-conversion image. D(0) represents image data in line No. 0 in the post-conversion image.
In the items shown in this figure, the five lines of the original image and the eight lines of the post-conversion image correlated, so the image data D of the post-conversion image naturally has spacing of ⅝=0.625 as compared to that of the image data of the original image. Representing the position of the image data of the post-conversion image by coordinates of the original image expresses D(0) as being 0.0, D(1) as 0.625, and D(2) as 0.625×2=1.250.
Now, looking more closely at the image data D(2) of the post-conversion image, the coordinate thereof is 1.250, at a position dividing the coordinate of the line data d(1) and d(2) of the original image by a ratio of 0.25 to 0.75. That is to say, in the event of using linear interpolation, for example, the image data D(2) of the post-conversion image can be obtained by D(2)=0.75×d(1)+0.25×d(2), from the image data d(1) and d(2) of the original image.
The line data D(0) and D(1) of the post-conversion image are calculated from the image data d(0) and d(1) of the original image. Also, D(2) and D(3) are calculated from d(1) and d(2). In the same way, D(4) is calculated from d(2) and d(3), and D(5) and D(6) are calculated from d(3) and d(4).
In the event that such a correlating relation between the line data of the original image and the line data of the post-conversion image are satisfied, scaling processing of the image is carried out normally. Incidentally, in the event of performing computation by image data on multiple lines as described above, a method is used wherein multiple lines of image data are read out using memory provided within the device.
FIG. 4 is a timing chart showing a case of performing image scaling processing in the vertical direction using line memory for three lines. This example also involves enlarging processing for converting five pixels into eight pixels, for the sake of explanation.
In the figure, the symbol (a) represents horizontal synchronizing signals of the input image signals, and (b) represents image data of the input image signals (i.e., original image data). The symbols (c), (d), and (e) represent write addresses and read addresses in the three lines of memory (wherein time passes in the direction of traveling right in the figure, and the address values of each increase with the passage of time).
The symbol (f) represents horizontal synchronizing signals of the output image signals. Now, let us say that the output image signals are image signals containing post-conversion image data. The symbols (g), (h), and (k) represent image data read out from the line memory.
In (a), Th represents the cycle of the horizontal synchronizing signals in the input image signals, and y, y+1, y+2, and so forth represent line position in the input image signals. In (b), d(y), d(y+1), d(y+2), and so forth represent image data corresponding to the line positions y, y+1, y+2, and so forth.
In (c), (d), and (e), the vertical axis represents addresses, with the dotted lines representing write addresses and the solid lines representing read addresses. In (f), Tid represents the cycle of the horizontal synchronizing signals in the output image signals, and Y, Y+1, Y+2, and so forth represent line position in the image signals of the post-conversion image.
The symbols (g), (h), and (k) represent image data read out from the line memory, each being output data of the operation of reading out the line memory of (c), (d), and (e).
The symbol (m) represents the results of performing interpolation filter processing to the image data read out from the line memory, i.e., image data of the post-conversion image. D(Y), D(Y+1), D(Y+2), and so forth represent image data corresponding to the line positions Y, Y+1, Y+2, and so forth.
The line memory writing operation is performed based on the horizontal synchronizing signal (a) of the input image signals. The image data d(y) of the line y is written to the line memory (e). At the point that the horizontal synchronizing signal from line y+1 is input, the line memory which is the object of writing thereof is switched from (e) to (c), and the image data d(y+1) is written to the line memory (c).
In the same way, the image data d(y+2) of the line y+2 is written to the line memory (d), and the image data d(y+3) of the line y+3 is written to the line memory (e). Subsequently, the writing operation of the image data d is performed while cyclically switching the line memory for each line.
In the figure, at the stage of the image data d(y) being written to the line memory (e), the image data d(y−2) is stored to the line memory (c), and the image data d(y−1) is stored to the line memory (d).
The read-out operation of the line memory is performed with the horizontal synchronizing signal (f) of the output image signals as the reference thereof, and operates such that the correlating relation between the original image data and the post-conversion image data is satisfied, as shown in FIG. 3. That is, at line Y, image data d(y−2) and d(y−1) are read out of the line memory (c) and (d) respectively, and output as shown by (g) and (h).
The image data (g) and (h) that have been read out are subjected to interpolation filter processing, thereby generating the image data D(Y) of the line Y in the post-conversion image. Looking more closely at the line memory that is read out, the reading out operation is performed from line memory (c) and (d) for line Y+1, from line memory (d) and (e) for line Y+2, from line memory (d) and (e) for line Y+3, from line memory (e) and (c) for line Y+4, from line memory (c) and (d) for line Y+5, from line memory (c) and (d) for line Y+6, and from line memory (d) and (e) for line Y+7.
Looking more closely at the image data read out, image data d(y−2) and d(y−1) is the image data read out at line Y+1, d(y−1) and d(y) at line Y+2, d(y−1) and d(y) at line Y+3, d(y) and d(y+1) at line Y+4, d(y+1) and d(y+2) at line Y+5, d(y+1) and d(y+2) at line Y+6, and d(y+2) and d(y+3) at line Y+7.
In order to perform the writing operation and reading operation of line memory, there is the need for the horizontal synchronizing signals (f) of the output image signals to occur at an appropriate cycle Tid. For example, in order to perform enlarging processing wherein five lines are converted into eight lines as described above, there is the need for the five lines of the input image signals and the eight lines of the output image signals to correlate.
That is to say, ideally, there needs to be a correlating relation of 5×Th=8×Tid between the horizontal synchronizing signal cycle Th in the input image signal and horizontal synchronizing signal cycle Tid in the output image signal. Hence, the ideal horizontal cycle of the output image signals can be obtained by Tid=Th×⅝.
Now, let us consider a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is shorter than the ideal horizontal cycle Tid.
FIG. 5 is a timing chart showing a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is Tid−α. For the sake of comparison, the timing of horizontal synchronizing signals of the ideal horizontal cycle Tid is shown above (f).
At the leading end of the line Y+1 of the output image signals, the leading end of the line is in a state of being shifted forwards timewise by an amount of α (i.e., a margin of error α has occurred as compared to the ideal horizontal synchronizing signal timing). Next, the margin of error α also occurs at the following line Y+2 as well, so the margin of error thus is 2α at the leading end of the line Y+2 of the output image signals. In the same way, the margin of error is 3α at the leading end of the line Y+3, and the margin of error α continues to accumulate for each subsequent line.
On the other hand, the horizontal synchronizing signal cycle Th on the input image signals remains the same. The timing of the writing operation of the line memory does not change, so the correlating relation between the line memory writing operation and the reading operation shifts away by an amount of α for each line of the output image signals.
Now, let us look more closely at the line Y+8 of the output image signals. A solid slanted line indicating the reading address of the line memory (c) and a dotted slanted line indicating the write address intersect at the time t1. The accumulated margin of error at this point is 8·α.
In the line Y+8 of the output image signals, the image data read out from the line memory (c) is d(y+4) written at line y+4 of the input image signals before the time t1, but is d(y+1) written at line y+1 of the input image signals after the time t1 (i.e., the hatched portion of (g).
In other words, after the time t1, image data three lines before the intended line is read out instead of the image data intended to be read out. Image data D(Y+8) which is the read out image data (k) and (g) which have been subjected to interpolation processing is abnormal data after the time t1 (i.e., the hatched portion of (m)). Thus, in such a case, the processing of scaling the image becomes impossible.
Note that hereafter, the phenomena of image data which should be read out being rendered unreadable due to the correlating relation between the writing operation and reading operation of the line memory shifting, will be referred to as “overtaking” on the line memory.
FIG. 6 is a diagram zooming in on the line memory (c) near the line y+4 of the input image signal in FIG. 5 and the read data (g) nearby. In FIG. 6, the upper part of the drawing shows a timing chart in a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is Tid (i.e., ideal cycle), and the lower part shows a timing chart in a case wherein the cycle of (f) is Tid−α.
The notation WA(y+4) represents the write address on the line memory at line y+4 of the image input signal. Now, RA(Y+6) RA(Y+8) represent the read address on the line memory at line Y+6 and line Y+8 at the upper part (ideal cycle). Also, RA′(Y+8) represents the read address on the line memory at line Y+8 at the lower part (horizontal cycle Tid−α).
As described above, RA′(Y+8) at the lower part has shifted forwards by an amount of time 8α as to RA(Y+8) at the upper part, and at the lower part the hatched portion of RA′(Y+8) and WA(y+4) intersect at the time t1. After time t1, the read out data (g) of the line memory (c) is not the image data d(y+4) which was intended to be read out, but rather the image data d(y+1) from three lines back.
At the upper part, the hatched portion of WA(y+4) and RA(Y+8) do not intersect and an interval (timing margin) of the time Tm1 exists at the timing of accessing the final address of each.
On the other hand, RA′(Y+8) at the lower part has shifted forwards by an amount of time 8α as to RA(Y+8) at the upper part, in the event that the accumulated margin of error 8α≧Tm1 as shown in the figure, the intended image data cannot be read. In order to correctly read the image data from the line memory, the accumulated margin of error for every lines of output image signals must always be smaller than Tm1.
That is to say, with the timing margin at line Y+Q as Tm(Q), image scaling processing will be impossible unless the accumulated margin of error Qα for Q lines satisfies the relation of Qα<Tm(Q).
Next, let us consider a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is longer than the ideal horizontal cycle Tid.
FIG. 7 is a timing chart showing a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is Tid+α. For the sake of comparison, the timing of horizontal synchronizing signals of the ideal horizontal cycle Tid is shown above (f).
At the leading end of the line Y+1 of the output image signals, a margin of error a occurs at the leading end of the line Y+1 of the output image signals, as compared to the ideal horizontal synchronizing signal timing, and the leading end of the line is in a state of being shifted backwards timewise by an amount of α. Next, the margin of error α also occurs at the following line Y+2 as well, so the margin of error thus is 2α at the leading end of the line Y+2 of the output image signals. In the same way, the margin of error is 3α at the line Y+3, and the margin of error a continues to accumulate for each subsequent line.
In this case also, the horizontal synchronizing signal cycle Th of the input image signals remains the same. The timing of the writing operation of the line memory does not change, so the correlating relation between the line memory writing operation and the reading operation shifts away by an amount of a for each line of the output image signals.
Now, looking more closely at the line Y+6 of the output image signals, the accumulated margin of error is 6α, and a solid slanted line indicating the reading address of the line memory (c) and a dotted slanted line indicating the write address intersect at the time t2.
In the line Y+6 of the image signals, the image data read out from the line memory (c) is d(y+4) written at line y+4 of the input image signals before the time t2 (i.e., the hatched portion of (g)), but is d(y+1) written at line y+1 of the input image signals after the time t2.
In other words, before the time t2, image data three lines after the intended line is read out instead of the image data intended to be read out. Image data D(Y+6) which is the read out image data (g) and (h) which have been subjected to interpolation processing is abnormal data before the time t2 (i.e., the hatched portion of (m)). Thus, in such a case as well, the processing of scaling the image becomes impossible.
FIG. 8 is a diagram zooming in on the line memory (c) near the line y+4 of the input image signal in FIG. 7 and the read data (g). In FIG. 8, the upper part of the drawing shows a timing chart in a case wherein the cycle of the horizontal synchronizing signal (f) of the output image signals is Tid (i.e., ideal cycle), and the lower part shows a timing chart in a case wherein the cycle of (f) is Tid+α.
The notation WA(y+4) represents the write address on the line memory at line y+4 of the image input signal. RA(Y+6) and RA(Y+8) represent the read address on the line memory at line Y+6 and line Y+8 at the upper part (ideal cycle).
Also, RA′(Y+6) and RA′(Y+8) represent the read address on the line memory at line Y+6 and line Y+8 in the event that the horizontal synchronizing signal cycle of the output image signals is Tid+α.
As described above, RA′(Y+6) at the lower part has shifted by an backwards amount of time 6α as to RA(Y+6) at the upper part, and at the lower part the hatched portion of RA′(Y+6) and WA(y+4) intersect at the time t2. After time t2, the read out data (g) of the line memory (c) is not the image data d(y+1) which was intended to be read out, but rather the image data d(y+4) from three lines later.
At the upper part, the hatched portion of WA(y+4) and RA(Y+6) do not intersect and an interval (timing margin) of the time Tm2 exists at the timing of accessing the leading end address of each.
On the other hand, RA′(Y+6) at the lower part has shifted back by an amount of time 6α as to RA(Y+6) at the upper part, and in the event that the accumulated margin of error 6α≧Tm2 as shown in the figure, the intended image data cannot be read. In order to correctly read the image data from the line memory, the accumulated margin of error per line of output image signals must always be smaller than Tm2.
Here as well, with the timing margin at line Y+Q as Tm(Q), image scaling processing will be impossible unless the accumulated margin of error Qα for Q lines satisfies the relation of Qα<Tm(Q).
From the above, it can be understood that in the event that the cycle of the horizontal synchronizing signal of the output image signals is longer than the ideal cycle Tid by an α amount of time, or in the event that the cycle is shorter than the ideal cycle Tid by an α amount of time, image scaling processing is impossible unless the accumulated margin of error Qα for Q lines of output image signals and the timing margin Tm(Q) at that line position satisfy the relation of Qα<Tm.
FIG. 9 is a diagram illustrating a conventional image display device disclosed in Japanese Unexamined Patent Application Publication No. 11-38955, for example. In the figure, reference numeral 101 denotes a PLL (Phase Locked Loop) circuit, 102 denotes an A/D converter, 103 denotes an oscillator, 104 denotes a read-start pulse generating circuit, 105 and 106 denote line memory, 107 denotes a scan converter, 108 denotes a D/A converter, 109 denotes a display unit, and 110 denotes a reset signal generating circuit.
Next, the operation thereof will be described. The PLL circuit 101 generates a write clock WCK synchronously with the horizontal synchronizing signal HSYNC obtained separated from the picture signals, and supplies this to the A/D converter 102 and the line memory 105 and 106.
The A/D converter 102 digitizes the input picture signals according to a predetermined sampling interval based on the write clock WCK from the PLL circuit 101, and supplies the picture data to the line memory 105 and the scan converter 107. The oscillator 103 generates read clocks RCK of a predetermined frequency, and supplies these to the read-start pulse generating circuit 104.
The reset signal generating circuit 110 generates a reset signal HRST each time five horizontal synchronizing signals HSYNC are counted, and supplies these to the read-start pulse generating circuit 104. The read-start pulse generating circuit 104 is reset each time reset signals HRST are supplied thereto, and generates read start pulses RDST and display start pulses HST each time the read clock RCK counts a predetermined number of counts.
The read start pulses RDST are supplied to the line memory 105 and 106, and the scan converter 107 and D/A converter 108. The display start pulses HST are supplied to the display unit 109.
FIG. 10 is a diagram illustrating the operation of the reset signal generating circuit 110 and the read-start pulse generating circuit 104. Here, the example of converting five lines into eight lines is used as an example. In the figure, a reset signal HRST is generated each time five horizontal synchronizing signals HSYNC are counted
A count value for the read clock RCK is set to the read-start pulse generating circuit 104, so as to allow generating of eight read start pulses RDST during the period of five horizontal synchronizing signals HSYNC.
Picture data is written to the line memory 105 in a manner synchronous with the write clock WCK from the PLL circuit 101, and data is read out in a manner synchronous to the read clock RCK from the oscillator 103 and the read start pulse RDST from the read-start pulse generating circuit 104, and the read out picture data is supplied to the line memory 106 and the scan converter 107.
Picture data written to the line memory 105 is written to the line memory 106 synchronously with the write clock WCK, picture data is read out in a manner synchronous with the read clock RCK and read start pulse RDST, and the read picture data is supplied to the scan converter 107.
The scan converter 107 performs predetermined interpolation processing to the picture data read from the line memory 105 and 106 respectively, and the picture data subjected to the interpolation processing is supplied to the display unit 109 via the D/A converter 108.
The display unit 109 is driven by the picture signals from the D/A converter 108 and the display start pulses HST from the read-start pulse generating circuit 104, and thus can display an enlarged picture.
Conventional image display devices and image processing devices have had the following problems, due to the above configuration.
As shown in FIG. 10, a count value is set for the read clock RCK at the read-start pulse generating circuit 104 so that eight read start pulses RDST can be generated during the period of five horizontal synchronizing signals HSYNC being input. The cycle of the read start pulse RDST ideally is ⅝ of the cycle of the horizontal synchronizing signal HSYNC.
Now, with the cycle of the read clock RCK as Tr, the ideal RDST cycle which is ⅝ of the horizontal synchronizing signal HSYNC cycle is not normally an integer multiple of the read clock cycle Tr, since the horizontal synchronizing signal HSYNC and the read clock RCK are asynchronous.
Here, an ideal read start pulse cycle can be expressed as (N+α)·Tr, wherein N represents an integer and a represents a decimal numeral, i.e., 0≦α1. These N and α represent cycle information, and in the following description, the integer N may be referred to as “First cycle information”, and α as “Second cycle information”.
However, the read start pulse RDST is generated by the read clock RCK, and accordingly the cycle of the read start pulse RDST can only be provided in integer multiples of the read clock RCK cycle Tr.
With the count value of the read clock RCK for generating the read start pulses RDST as N, the cycle of the read start pulse RDST for seven lines following the reset signal HRST is N·Tr, and is shorter than the ideal RDST cycle (N+α)·Tr by α·Tr.
Accordingly, the read start pulse RDST shifts away from the ideal RDST cycle by a margin of error of α·Tr by each line. That is, seven lines worth of margin of error are accumulated after seven lines following the reset signal HRST, so an accumulated margin of error of 7·α·Tr occurs at the leading end of the eighth line following the HRST.
Then, the next reset signal HRST resets the timing of generating read start pulses RDST at the read-start pulse generating circuit 104. Accordingly, the accumulated margin of error of 7·α·Tr is corrected here at the time of resetting. Or, conversely, it can be said that the accumulated margin of error will continue to increase until the generation timing of the read start pulses is corrected by the reset signal HRST.
As already described with reference to FIGS. 6 and 8, in the event that the reading operation shifts off of the line memory writing operation by a timing margin of Tm(Q) or more, proper scaling processing of the image becomes impossible.
With the example shown in FIG. 10, image scaling processing can be correctly performed in the event that the accumulated margin of error of 7·α·Tr of the generation timing of the read start pulse RDST is less than the time margin Tm, i.e., in the event that 7·α·Tr<Tm holds. However, in the event that 7·α·Tr≧Tm holds, proper scaling processing of the image is impossible.
In the event of converting a 480-line image into a 768-line image, the conversion ratio is 480:768, which translates into 5:8, so this processing is an operation of repeatedly converting five lines into eight lines. That is to say, the maximum accumulated margin of error is 7·α·Tr.
However, considering a case of converting a 480-line image into a 767-line image, the conversion ratio of 480:768 cannot be expressed in smaller integers, so the read-start pulse generating circuit 104 operates so as to generate 767 read start pulses RDST in a period of 480 horizontal synchronizing signals HSYNC.
The read-start pulse generating circuit 104 must generate 767 lines of read start pulses before the correction of the timing of generating the read start pulses RDST. In this case, the accumulated margin of error has a maximal value of 766·α·Tr. Depending on the value of α, 766·α·Tr will exceed the timing margin Tm(Q), and proper scaling processing of the image will become impossible.
Comparing the maximum accumulated margin of error of the conversion ratio of 5:8 with that of the conversion ratio of 480:767, the latter is more than 100 times the former, so the probability of the latter exceeding the timing margin Tm(Q) is great. That is, there are scaling ratios which allow image scaling processing, and scaling ratios which do not.
Accordingly, conventional image display devices have hand the problem that image scaling processing could not be performed at arbitrary conversion ratios using a clock which is asynchronous with input image signals.
FIGS. 11A and 11B are diagrams illustrating an example of image display with a conventional image display device. With the conventional image display device, the conversion ratio (conversion ratio) cannot be arbitrarily set, so only conversion images smaller or greater than the display area size of the display panel can be generated.
FIG. 11A is a display example wherein the conversion image size is smaller than the display area. Here, an invalid area (a) wherein no image is displayed occurs on the display area. FIG. 11B is a display example wherein the conversion image size is larger than the display area. Here, here is the problem that there is an area (b) wherein the conversion image cannot be displayed.